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Abstract 

The Framework for School Leaders, an architecture derived from the Interstate School Leaders Li¬ 
censure Consortium (ISLLC) Standards, is utilized in the design of the Principal Internship Mentor’s 
Assessment (PIMA). PIMA outcomes are reported for average scores for each standard and investigated 
as a measure of ISLLC Standards achievement and for predictive scores on the School Leaders Licensure 
Assessment (SLLA). Findings include consistency of variance within the PIMA and its constituent items. 
No positive relationships were found between SLLA scores and individual PIMA items. In addition, find¬ 
ings indicate reliability for the instrument. 
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2 Sumario en espanol 

La Armazon para Educa a Lideres, una arquitectura derivada de la Escuela Interestatal Lideres Licensure 
Consorcio (ISLLC) Estandares, son utilizados en el diseno de Evaluacion del Principal Puestos de interno 
Mentor (PIMA). Los resultados de PIMA son informados para cuentas medias para cada estandar e investi- 
gados como una medida de logro de Estandares de ISLLC y para cuentas predictivas en la Escuela Lideres 
Licensure Evaluacion (SLLA). Las conclusiones incluyen consistencia de variation dentro del PIMA y sus 
articulos constituyentes. Ningunas relaciones positivas fueron encontradas entre cuentas de SLLA y articulos 
individuales de PIMA. Ademas, las conclusiones indican la c.erteza para el instrumento. 

NOTE: Esta es una traduction por computadora de la pagina web original. Se suministra como 
information general y no debe considerarse completa ni exacta. 


3 Introduction 

“The internship is considered by many practicing principals to be the most valuable component of their 
preparation program, as schools provide the laboratories where the connection between educational leadership 
theory with practice and application can best be made” (Bost, 2009, p. 15). In the quest to build and 
maintain high quality educational leadership preparation “all programs struggle with ways to provide robust 
internships” (Darling-Hammond, Myerson, LaPoint, & Orr, 2010, p. 74). Finding time for assessing the 
intern is among the greater issues facing busy mentoring principals in the field. Few have time for an elaborate 
evaluation process. Despite this reality, university internship supervisors seek evidence that the intern 
has met sufficient “performance of core school leaders responsibilities based on clearly defined performance 
standards” (Fry, Bottom, & O’Neill, 2005, p.7). 

The instrument used for assessing interns becomes a critical component of the principal preparation 
program and requires evidence that it is linked to clearly defined performance standards. The Interstate 
School Leaders Licensure Consortium (ISLLC) Standards are recommended for performance standards in 
school leadership preparation programs (Teacher Education Accreditation Council, 2005). ISLLC standards 
are utilized for items on this study’s internship assessment instrument. 

4 Literature Review 

The ISLLC Standards “effectively reflect the wide variety of responsibilities associated with the principalship” 
(Bost, 2009, p. 5). “There is no consistent set of program strategies in place across the country” (Darling- 
Hammond, Myerson, LaPointe, & Orr, 2010, p. 159). By utilizing the ISLLC framework for assessing 
program outcomes, states are “beginning to create more systematic approaches” (p. 159). “ISLLC standards 
have been used for guidelines in principal preparation programs and they have sharpened the focus of 
principal training considerably” (p. 150). For this reason, the ISLLC standards were selected for the 

Principal Internship Mentor’s Assessment (PIMA) examined in this study. 

The Regent University Education Leadership Program adopted the six ISLLC standards for learning 
outcomes. These standards were also selected by Program faculty for national accreditation with the Teacher 
Education Accreditation Council (TEAC). The Program was accredited on January 9, 2009. TEAC is 
currently one of two national accreditors currently recognized by the U.S. Department of Education (USDE) 
and by the Council of Higher Education Accreditation (CHEA) (NCATE/TEAC, 2010). The other national 
accreditor is the National Council for Accreditation of Teacher Education (NCATE). NCATE adheres to 
the Educational Leadership Constitutes Council (ELCC) standards for programs in educational leadership, 
ISLLC and ELCC standards are identical except that ELCC standards have an additional seventh standard; 
7.0 internship (National Policy Board for Educational Leadership, 2002). 
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4.1 

Standard 7.0 Narrative Explanation: This standard addresses the importance of structured, sustained, 
standards-based experiences in authentic settings. The internship is defined as the process and product 
that results from applying the knowledge and skills described in the previous standards in a workplace envi¬ 
ronment. Application of standards based knowledge, skills, and research in real settings over time is a critical 
aspect of any institutional program. The provision of graduate credit allows institutions to underscore the 
importance of this activity (p. 18). 

States vary greatly in their program internship requirements. For example, “92 percent of New York’s 
principals had had an internship as part of their program preparation, whereas only 8 percent of Delaware’s 
principals had” (Darling-Hammond, Myerson, LaPointe, and Orr, 2010, p. 152). Internships across the 
country “range from a certain number of acquired credit hours that can be satisfied at least partially at a 
candidate’s home school or during school breaks, to year long, paid assignments working side by side with 
a mentoring principal” (Martin, Wright, Danzig, Flanary, & Brown, 2005, p. 134). It is likely there are as 
many internship assessments as there are different leadership preparation programs in the nation. Since no 
evidence was found in the literature for a common principal internship assessment, there are likely a large 
number of different instruments. 

There are examples of assessments for principal internships in the literature. Dallas Baptist University 
(2004) employs an instrument titled “Supervising Administrator Evaluation Form” (p. 14). This assessment 
is brief, employs the ISLLC standards and a Likert-type scale. It is very similar, although not as extensive 
an instrument as the one being addressed in this study. In Dallas’s instrument there is only one indicator 
for each of the six ISLLC standards. An instrument using ISLLC standards can be found in School Lead¬ 
ership Internship (Martin, Wright, Danzig, Flanary, & Brown, 2005), but this assessment focuses only on 
dispositions. 

Another instrument, MSEd (Principal, Building Level) Practicum Internship Assessment, from the Uni¬ 
versity of Nebraska at Kearney (2003) is a one-page checklist of 13 questions and one open-ended question. 
It does not appear to be aligned with ISLLC nor any other standards. Florida Atlantic University (2009) 
uses a Principal Internship Activity Log, with a box for standards on the form, where students fill in the 
log and the mentor initials and can make comments. Other principal intern assessments reviewed were more 
comprehensive and included: reflective journals and journey mapping (Cooner & Dickmann, 2006); profi¬ 
ciencies exhibited and explained in the Applied Principal’s Portfolio (Indiana Wesleyan University, 2010); 
work log and mentor’s assessment (Cunningham, 2005). 

5 A Framework Derived from the ISLLC Standards 

This study makes application of “The Framework for School Leaders”, an architectural design derived from 
the ISLLC Standards (Hessel & Holloway, 2002, p. 26). The Framework translates theory into practice and 
defines the elements of appropriate practice for school leaders (Hessel & Holloway, 2002). The Framework 
presents each of the six standards as a set of “four components of professional practice for school leaders, 
or specific aspects of the Standards” (p. 26). The Principal Internship Mentor’s Assessment (PIMA) is the 
result of synthesizing the framework into an instrument used to assess interns by mentoring principals in 
the field (Appendix A). The goal for the university program in this study was the design of an assessment 
that was researched-based, linked to the ISLLC standards, and practical, in terms of time, for mentoring 
principals completing the assessment. 

A rubric (see Appendix B), designed to identify performance levels and specific activities for scoring 
each of the four components in each ISLLC standard, provides a common set of parameters for principals 
completing the assessment. The PIMA contains five performance levels displayed in a Likert-type scale 
within the assessment. 

The PIMA is utilized as a summative evaluation for the internship. “These performance indicators are 
used by the university instructor to designate the intern’s level of performance on each of the components of 
the dimensions of school administrative factors” (Cunningham, 2007, 22). The outcomes provide evidence 
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required for a grade in the principal internship course and data for accreditation evidence and program 
improvement. 

6 The School Leaders Licensure Assessment 

The School Leaders Licensure Assessment (SLLA) “measures whether entry-level education leaders have the 
standards relevant knowledge believed necessary for competent professional practice” (Education Testing 
Services, 2010, p. 1). The content of the SLLA is “keyed to the ISLLC standards and the knowledge, skills, 
dispositions, and performances they reflect” (Jones & Kennedy, 2008, p. 5). This study examines scores prior 
to the new revised assessment that went into effect September 2009. The format for the six-hour assessment 
utilized in this study is divided into four sections: 

• Evaluation of Actions I (1 hour) - Ten short vignettes covering situations a principal might encounter. 
Candidates respond to a focused question that asks for next steps, factors influencing a decision, or 
possible consequences of an action. 

• Evaluation of Actions II (1 hour) - Six longer vignettes. Each presents a dilemma based on typical 
school issues. Candidates analyze the circumstances and respond to a focused analytical question that 
requires prioritizing action steps or articulating the relevant issues. 

• Synthesis of Information and Problem-Solving (2 hours) - Two case studies involving teaching and 
learning issues. In each case, candidates propose a course of action to address a complex problem, 
referring to a set of documents, and a short scenario describing a school and its community. 

• Analysis of Information and Decision Making (2 hours) - Seven documents that relate to teaching and 
learning issues. Candidates answer two questions about each document (Virginia Tidewater Consor¬ 
tium For Higher Education, 2005, pp. 12 & 13). 

Situations presented on the SLLA are designed to be real-life in the day-to-day activities of a principal. The 
examinee responds to each question and is evaluated with a scoring rubric grounded in the ISLLC standards. 

Many states use the SLLA for licensure or endorsement in educational administration and utilize a 
cut score for candidates to attain before a license/endorsement/certificate is awarded. Many university 
educational administration preparation programs use candidate scores on the SLLA as a measure of program 
success. 

6.1 Statement of Purpose 

The purpose of this study is to examine the utility of PIMA as a measure of ISLLC Standards achievement, 
and as a predictor of SLLA scores. The ultimate outcome is improving principal preparation programs as 
required by accrediting agencies and state program approval regulations. 

6.2 Research Question 

1. Can performance on the PIMA predict success as measured by the SLLA? 


7 Methods 

7.1 Participants 

Fifty-nine Educational Leadership program completers were studied. The participants were uncompensated 
and were not interviewed, tested or surveyed beyond the normal program requirements. All were licensed and 
experienced educators prior to program commencement. The participant group consisted of all personnel 
that completed the leadership program including internship and who also took the SLLA between September 
2005 and August 2009. The final data draw for this study was August 2009 because a new SLLA began in 
September 2009. 
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7.2 Apparatus 

The PIMA is a 24-item Likert-type scale instrument derived from the ISLLC standards. There are four 
items per standard with each item being rated on a 5-point Likert-type scale (see Figure 1). 

Figure 1. Performance Levels of the PIMA Likert Scale 


U-lmls l: Address*-* ev Groce :-1 «no«rGgr. j-Gr"sta-G -g. a-a:or ate- cat*- 

1- v 0 gua/ahielel 0 l evidence of knowledge. understanding, andtof epplicalidn 

2 - Issb dels ;sc-rah: evidence of knowledge understanding. and/or appllcalion 
^-d-E-lailedi'spectfflc ewdeoce of snowl-sdge, understanding, a-G.'or a -:at 
4-vary Ez-=-:if ci'-ionv/’cing evidence of knowledge, understand, andtor applicalion 


The four items per standard are taken from “Components of Professional Practice for School Leaders” 
(Hessel & Holloway, 2002, p.27) (see Figure 2). The rating form reflects the 24 Components of Professional 
Practice for School Leaders derived directly from the ISLLC standards (Hessel &Holloway, 2002, p. 27). Each 
item was directly linked to an ISLLC standard. The items and the instrument were reviewed for content 
validity by the program faculty. The form was piloted in the 2006-2007 academic year. The Educational 
Focus Group (Cannizzaro, 2007) provided feedback on the form and confirmed its content validity since 
participants were practicing experts in the field. The Pilot Study consisted of interns that completed an 
internship and a recorded score for the SLLA (N = 4). The investigators reasoned that the small sample and 
low variability among scores hampered statistical inference. The N (N=4) is small because the University 
did not require an internship before May 2007. To address inter-rater reliability, sets of two raters used the 
form and discussed the outcomes in the Focus Group and the four teams of practitioners rated the PIMA 
similarly (Cannizzaro, 2007). Subjective scoring (Inter-rater-reliability/consistency between tests) is helped 
when usable guidelines for scoring are developed such as the scoring rubric for the PIMA. 

7.3 Procedure 

Collection of data included principals’ completion of the PIMA each semester for each student completing 
an internship. The final date for collection of data was August 2009 because the new SLLA format began in 
September 2009. These data was reported in an ongoing recorded history of program internship data. PIMA 
scores were extracted from this spreadsheet. SLLA scores were extracted from the university data base. A 
Pearson’s correlation was performed for average scores for each ISLLC Standard. Another correlation was 
performed using each individual item and SLLA scores. 

Figure 2. Components of Professional Practice for School Leaders 
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Standard 1: The Vision of Learning 

la. Developing the Vision 

lb. Communicating the Vision 

lc. Implementing the Vision 

l d. Monitoring and Evaluating the Vision 

Standard 2: The Culture of Teaching and Learning 
2a. Valuing Students and Staff 
2b. Developing and Sustaining the Culture 
2c. Ensuring an Inclusive Culture 
2d. Monitoring and Evaluating the Culture 

Standard 3: The management of Learning 
3a. Making Management Decisions 

3b. Developing Procedures to Ensure Successful Teaching and Learning 

3c. Allocating Resources to Ensure Successful Teaching and Learning 

3d. Creating a Safe. Healthy Environment to Ensure Successful Teaching and Learning 

Standard 4: Relationships with the Broader Community to Foster Learning 
4a. Understanding Community Needs 
4b. Involving Members of the Community 

4c. Providing Opportunities for the Community and School to Serve Each Other 
4d. Understanding and Valuing Diversity 

Standard 5: Integrity. Fairness, and Ethics in Learning 

5a. Demonstrating a Personal and Professional Code of Ethics 
5b. Understanding One's Impact on the School and Community 
5c. Respecting the Rights and Dignity of All 
5d. Inspiring Integrity and Ethical Behavior in Others 

Standard 6: The Political. Social, Economic, Legal, and Cultural Context of Learning 
6a. Operating Schools on Behalf of Students and Families 
6b. Communicating Changes in Environment to Stakeholders 
6c. Working Within Policies. Laws, and Regulations 
6d. Communicating with Decision-Makers Outside the School Community 


8 Results 

Overall, the implementation of PIMA was consistent with the program’s standards. Internship mentors are 
defined as principals practicing in the field during the time of the internship. The numbers of students who 
completed internships (N = 78) is relatively small since internships were not required until August, 2007 
(Virginia Department of Education, 2007). Additionally, SLLA scores were not available for some individuals 
bringing the number of students available for study down to 59 participants. Aggregate data indicate that 
all average scores exceeded the 75% benchmark (3.00) set by the program faculty. The faculty reset the 
benchmark to 80% in 2008. Results are reported in Table 1. 
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Summary of Average Scores for Each ISLLC Standard 2005-2010 











la 

lb 

lc 

Id 




Mean 

3.68 

3.66 

3.64 

3.69 

Total Average of Standard: 

3.66 


S.D. 

0.54 

0.49 

0.51 

0.49 

Total Average of Deviation: 

0.54 











2a 

2b 

2c 

2d 




Mean 

3.81 

3.67 

3.70 

3.68 

Total Average of Standard: 

3.71 


S.D. 

0.43 

0.54 

0.49 

0.50 

Total Average of Deviation: 

0.49 











3a 

3b 

3c 

3d 




Mean 

3.71 

3.69 

3.60 

3.75 

Total Average of Standard: 

3.66 


S.D. 

0.52 

0.50 

0.53 

0.51 

Total Average of Deviation: 

0.51 











4a 

4b 

4c 

4d 




Mean 

3.76 

3.67 

3.64 

3.76 

Total Average of Standard: 

3.71 


S.D. 

0.42 

0.47 

0.65 

0.44 

Total Average of Deviation: 

0.49 











5a 

5b 

5c 

5d 




Mean 

3.81 

3.75 

3.81 

3.83 

Total Average of Standard: 

3.80 


S.D. 

0.39 

0.60 

0.47 

0.42 

Total Average of Deviation: 

0.59 











6a 

6b 

6c 

6d 




Mean 

3.67 

3.59 

3.54 

3.53 

Total Average of Standard: 

3.58 


S.D. 

0.66 

0.57 

0.65 

0.58 

Total Average of Deviation: 

0.59 



Table 1 


note: 100% of candidates meet or exceed score of 3.00; 3.2% after 2008. 

Table 1 reports the average scores from the mentoring principal’s assessment for each of the four items under 
ISLLC Standards 1-6. All average scores exceeded the 75 % (3.00); 80% (3.20) after 2008 standard set by 
the program faculty. 

The investigators performed a correlation analysis on mean scores for each of the six standards, the 
overall PIMA and scores on the SLLA. There were strong to very strong, statistically significant relationships 
between the means for each of the 6 standards and the PIMA overall (see Table 2). 
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Pearson Correlations among ISLLC Standard sections, the PIMA and SLLA Scores 












Standard 2 

Standard 3 

Standard 4 

Standard 5 

Standard 6 

PIMA 

SLLA 











Standard 1 

.666** 

.651** 

.704** 

.640** 

.813** 

.856** 

.072 


Standard 2 


.721** 

.801** 

.769** 

.663** 

.876** 

-.002 


Standard 3 



.727** 

.703** 

.673** 

.845** 

.065 


Standard 4 




.758** 

.777** 

.904** 

.001 


Standard 5 





.722** 

.857** 

-.053 


Standard 6 






.892** 

-.004 


PIMA 







-.029 



Table 2 


NOTE: A second correlation study was performed using SLLA Scores and the individual responses 
on the PIMA. As with the correlation noted for the means by standard, moderate to strong rela¬ 
tionships were found between individual items. 

For each standard the constituent items correlated robustly with the standard overall. Pearson’s r values 
ranged between .779 and .980. All findings fell within the .01 level of significance. It was also noted that 
individual items correlated significantly with the other items grouped within the same standard. 

9 Discussion of Results 

The average scores for each standard ranged from 3.58 to 3.80 on a scale from 0.00 to 4.00. Standard 
deviations for the standards fell between 0.49 to 0.59. The strong to very strong relationships evidenced by 
the Pearson’s correlation analysis indicate that the six mean standard scores as measured by the PIMA are 
consistent in variation. This is also true of the individual items with each standard. 

The precise relationship between the ratings on the PIMA and SLLA scores is of primary interest to 
the researchers. Although a relationship seems likely, this initial analysis lacked the statistical power to 
support the research hypothesis. The apparent internal consistency and significant relationship between 
PIMA individual items and standards suggests that further study is warranted. The researchers recommend 
the use of the PIMA in a longitudinal study to further describe the relationship between PIMA ratings and 
SLLA Scores. 

One limitation of the study is that interns completed a formal leadership preparation program designed 
primarily with the ISLLC Standards as the foundation and external validity may not be extended beyond 
interns is this particular program. There may be some generalization to other programs that use the ISLLC 
as the basis for their performance standards. Because the findings are limited in scope, the researchers 
recommend additional study using the ISLLC Standards to examine if similar or more favorable results can 
be obtained. 

This study is important in the use of internships and particularly standards based measures used in 
internship assessment. Literature suggests that although not universal, internships are beneficial and should 
be robust. This includes the assessment of the intern by mentoring principals. The contribution of this study 
is to assist practitioners, both principals and university supervisors in the field, to better assess interns. It 
is recommended that the program continue to utilize the PIMA and compare the findings to the new SLLA 
outcomes that, as of fall 2009, report scores on each individual ISLLC Standard. Although adjustment of the 
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PIMA is justified, the researchers suggest the PIMA construct also be used as the basis of further research. 
This will enhance validity of the instrument and provide outcomes that can be applied to specific program 
improvement. 


10 Conclusions 

Since the researchers are interested in validity and reliability of the instrument, it is recommended that the 
PIMA be reviewed for redesign. A change in Likert-type scale would eliminate Zero and Not Applicable and 
would change to a true Likert scale, using the common 1 to 5 intervals. Additionally, the wording of the items 
may be altered to reflect the mentor’s perception of the intern’s ability to do a task rather than assessment 
of actual task performance. The wording of PIMA items would be changed to reflect how candidates would 
make decisions as future building principals. This is aligned with the measures found in the School Leaders 
Licensure Assessment (SLLA) where candidates simulate decision making. 

Recommendations for further study include: when feasible, have enough numbers to do a confirmatory 
factor analysis; a comparison of PIMA data triangulated with a university supervisor assessment, and a 
self-assessment utilizing the same assessment items; comparisons of results in this program with results in 
another university preparation program; and, once the PIMA is revised, compare results with half the sample 
using the original PIMA and half using the new version of the PIMA (see Figure 3). 

A final recommendation is to repeat the study utilizing the revised Principal Internship Mentor’s Assess¬ 
ment and the new SLLA format that went into effect on September 2009. The new format is shorter (4 hours 
as compared to 6 hours in the previous format). The new format also has been reduced to two sections. 

10.1 

Section I contains 100 multiple-choice questions and Section II, seven constructed- response questions that 
call for written answers based on scenarios and sets of documents that an education leader may encounter. In 
answering the questions, candidates are required to analyze situations and data, propose appropriate courses 
of action, and provide rationales for their proposals. (Education Testing Services, 2010, p. 1) 

Figure 3. Principal Internship Mentor’s Assessment Revised for Standard 1 

PERFORMANCE LEVELS FOR EACH “STANDARD”: 

1- fails to address/no evidence of knowledge, understanding, and/or application 

2- vague/skeletal evidence of knowledge, understanding, and/or application 

3- less detail/specific evidence of knowledge, understanding, and/or application 

4- detailed/specific evidence of knowledge, understanding, and/or application 

5- very specific/convincing evidence of knowledge, understand, and/or 
application 

Standard 1: A school administrative Intern is an educational leader who promotes the 
success of ai students by tac«totlng the development, articulation. Imp ementatlon, ana 
stewardship of a vision of learning tnaf is shared and supported by the school community. 

1 a. Developing the vision 1 2 3 4 5 

lb. Communicating the vision 1 2 3 4 5 

lc. implementing the vision 1 2 3 4 5 

1 d. Monitoring and Evaluotlng the Vision 1 2 3 4 5 

Comments:_ 
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The first section is completed by the candidate on-line and the seven constructed response questions are 
completed by the candidate at a testing center. The examinee responds to each multiple choice question 
(vignettes) and each constructed response question (case-studies) and is “evaluated with a scoring rubric 
grounded in ISLLC standards” (p. 5). 

Click Here for Appendix A: Principal Internship Mentor’s Assessment 2 

Click Here for Appendix B: Principal Internship Mentor’s Assessment (PIMA) Rating Rubric 3 
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http: //cnx.org/content/m36788/l.l/ 
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